Correspondence Measures for MT Evaluation

A b s t r a c t The paper presents a descriptive model for measuring the salient traits and tendencies of a translation as compared with the source text. We present some results from applying the model to the texts of the Linköping Translation Corpus (LTC) that have been produced by different kinds of translation aids, and discuss its application to MT evaluation. In this paper we present a correspondence model and a set of measures that are of interest to linguistic evaluation of translations, including MT evaluation. As a point of departure we contend that the methodology of linguistic evaluation of machine translation systems must be set within the more general framework of evaluating translations, irrespective of the means and processes used to produce them. A counter-argument to this view is that MT output, because of its deficiencies, serves different functions than translations produced by humans. A strong defender of this view is Sager (1993) who argues that MT output is best treated as a text type of its own " with a limited range of possible functions " (op.cit. p. 263) and that MT evaluation should focus on the appropriateness of output for the intended use. Of course there can be no denial that machine translation of today has limitations and that it is often used for other purposes than human translations, such as reading assistance of foreign language texts and cross-language information retrieval. Current developments in MT technology and MT use, however, make it more and more difficult to uphold a strict division between MT and HT. On the one hand, corpus-based methods such as statistical and example-based MT try to make use of human translation strategies as they are manifested in existing translations; thus it is of great interest to see how far these methods are able to succeed. On the other hand, MT is more and more used as a component in larger systems; for example in machine-aided human translation where a translation memory component, a term bank and various other tools are included. Again, with such a setup it is of interest to know how the end product compares to the translations produced by human translators without specific translation aids. The correspondence model is aimed at describing and quantifying structural and semantic relations between source text and translation; a common practice in descriptive translation studies (cf. e.g. van Leuven-Zwart, 1990). Thus, the goal …